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Abstract 

An algorithm that performs joint equalization and decoding for nonlinear two-dimensional intersym- 
bol interference channels is presented. The algorithm performs sum-product message-passing on a factor 
graph that represents the underlying system. The two-dimensional optical storage (TWODOS) technology 
is an example of a system with nonlinear two-dimensional intersymbol interference. Simulations for 
the nonlinear channel model of TWODOS show significant improvement in performance over uncoded 
performance. Noise tolerance thresholds for the algorithm for the TWODOS channel, computed using 
density evolution, are also presented and accurately predict the limiting performance of the algorithm as 
the codeword length increases. 

Index Terms: Low-density parity-check codes, optical storage, sum-product algorithm, TWODOS, two-dimensional 
intersymbol interference. 

I. Introduction 

Two-dimensional (2D) intersymbol interference (ISI) channels have received a lot of attention lately. This is 
mainly due to the fact that research focus in storage is shifting towards developing a two-dimensional storage 
paradigm. Although conventional recording media like magnetic hard disks and DVDs use planar storage, the 
second dimension is utilized only loosely. Significant increase in storage density can be obtained by moving towards 
truly two-dimensional storage. Patterned magnetic media [1], holographic storage [2], and two-dimensional optical 
storage [3], [4] (TWODOS) are examples of upcoming technologies that use a two-dimensional storage paradigm 
and promise terabit storage density at high data rates. Due to the two-dimensional nature of storage, these advanced 
storage technologies have 2D ISI during the readback process. Conventional methods like partial response maximum- 
likelihood decoding, that have proved very successful for one-dimensional ISI channels, do not extend to two 
dimensions. This motivates the need for new methods to combat 2D ISI. Besides advanced storage technologies, 
multi-user communication scenarios, like cellular communication, also have situations where 2D ISI is prevalent. 
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Several detection (or equalization) algorithms for 2D ISI channels have been proposed [5-11]. Some of these 
algorithms are based on linear minimum mean-square-error (MMSE) equalization; the MMSE equalization is 
followed by (soft or hard) thresholding and may iterate between equalization and thresholding. Others are based on 
multi-track versions of the Viterbi or BCJR algorithm accompanied by decision feedback. Singla et al, [1 1-14] have 
proposed joint equalization and decoding schemes for 2D ISI channels and have shown the benefit of using error 
control coding in conjunction with detection. More often than not, the ISI is modeled as a linear filter. Although a 
good starting point, the linearity assumption doesn't hold in general. TWODOS is an example of a system where 
the ISI is nonlinear. 

TWODOS is, potentially, the next generation optical storage technology with projected storage capacity twice 
that of the blu-ray disk and with ten times faster data access rates [3], [4]. As in conventional optical disk recording, 
bits in the TWODOS model are written on the disk in spiral tracks. However, instead of having a single row of 
bit cells, each track consists of a number of bit rows stacked together making TWODOS a truly two-dimensional 
storage paradigm. Successive tracks on the disk are separated by a guard band which consists of one empty bit 
row. In addition, the bit cells are hexagonal; this allows 15 percent higher packing density than rectangular bit 
cells leading to even higher storage capacity. As in conventional optical disk recording, a 0/1 is represented by 
the absence/presence of a pit on the disk surface. A scalar diffraction model proposed by Coene [3] for optical 
recording is used to model the readback signal from the disk. Under this model the readback intensity from the 
disk has linear and bilinear contributions from the stored data bits. 

Various detection schemes for TWODOS have been proposed [15-17]. These schemes, with the exception of 
Immink et al, [15], use two-dimensional partial response equalization to obtain a linear channel model for the 
ISI. Then, equalization methods, like MMSE equalization, are used for detection on this linearized channel model. 
Since partial response equalization leads to noise correlation there is an inherent loss associated with these schemes. 
Thus, it is prudent to search for decoding schemes that avoid partial response equalization and are designed taking 
into account the nonlinear structure of the ISI. Immink et al, [15] propose using a stripe-wise Viterbi detector 
that is designed for the nonlinear ISI. Chugg et al, also proposed equalization schemes for nonlinear 2D ISI 
channels [5], [6]. However, neither of the aforementioned schemes employed error control coding. 

In this paper, a low-complexity scheme for joint equalization and decoding for nonlinear 2D ISI channels is 
presented. The scheme was first proposed for linear 2D ISI channels [12] and has been appropriately modified for 
the nonlinear channel. This scheme, called the full graph scheme, performs sum-product message-passing on a joint 
graph that represents the error control code and the nonlinear 2D ISI channel. Low-density parity-check (LDPC) 
codes [18] are used for error correction. Simulations for the nonlinear channel model of TWODOS demonstrate 
the potential of using the full graph scheme. Significant improvement in performance is observed over uncoded 
performance. Noise tolerance thresholds are calculated for regular LDPC codes of different rates and sum-product 
decoding for the nonlinear 2D ISI channel. We note that others have also proposed message-passing based schemes 
for joint equalization and decoding for a wide variety of channels like the fading channel [19] and partial response 
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channels [20], [21]. 

The paper is organized as follows. The system model and the channel model for TWODOS is described in 
Section|n] The full graph message-passing algorithm and its performance for TWODOS are presented in Sectionim 
The density evolution algorithm and the noise tolerance thresholds are presented in Section HVl SectionlVlconcludes 
the paper. 

II. System model 

The system is modeled as a discrete-time communication system; 

r{i,j) = h({x(k,l) : {k,l)GAfij}) + w(i,j), (1) 

where r(i,j) are the data received at the output of the channel; x(k, I) are the channel inputs, obtained by encoding 
the user data with an error control code; w(i,j) are samples of additive white Gaussian noise (AWGN) with zero 
mean and variance a 2 and are assumed to be independent of x(k,l); Mij is the set of indices of all the bits that 
interfere with x(i,j) during readback, including x(i,j); and h(-) is the function that encapsulates the nonlinear 2D 
ISI. The user data and the encoded data are assumed to be binary. LDPC codes are used for error correction. 

For TWODOS, a scalar diffraction model proposed by Coene [3] for optical recording is used to model the 
readback signal. Using the model, the readback signal (optical intensity) from the disk can be written as 

r(i, j) = 1 — YJ Cij(k, l)x(k, I) + dij(k,l;m,n)x(k, l)x(m,n) +w(i, j), (2) 

(k,l) (fc,i)/(m,n) 

where aj(k,l) and dij(k,l;m,n) are the linear and nonlinear ISI coefficients, respectively. These coefficients 
depend on the parameters of the optical system, such as the wavelength of the laser, numerical aperture of the 
readback lens and geometry of the recording (pit and track dimensions). The extent of the interference is limited by 
the spot size of the read laser. Typically, this restricts the interference to nearest neighbors only. As shown in [3] 
this assumption is quite accurate. 

Using a nearest neighbor interference model, the signal intensity in (|2ji depends on the data bit stored in the central 
bit cell and the 6 neighboring bit cells. If it is assumed that two configurations with the same central bit and same 
number of nonzero neighbors have identical signal values then the signal intensity takes on 14 values corresponding 
to the 14 different configurations. As shown in [3], this symmetry assumption is a good approximation. Fig^shows 
four of these 14 configurations. 

Table U lists the signal levels for the 14 different configurations for one choice of pit dimensions and laser spot 
size. This table is reproduced from [3]. Looking at the table, the nonlinearity of the ISI is quite apparent; the signal 
level does not change linearly as the number of nonzero neighbors increases and the range when the central bit is 
a is greater than when the central bit is 1. 
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Fig. 1. Four of the possible 14 nearest neighbor configurations for TWODOS. The dark circles in the cells depict a pit 
corresponding to a stored 1. Absence of a pit indicates a stored 0. The pits cover only about half the area of the hexagonal bit 
cells. This is done to reduce signal folding [3]. 

TABLE I 

Signal levels for TWODOS recording using nearest neighbors interference. (Reproduced from [3]) 



Nonzero 


Central 


Central 


neighbors (n) 


bit=0 (Sn ) 


bit=l (s ni ) 





0.95 


0.50 


1 


0.80 


0.35 


2 


0.70 


0.30 


3 


0.55 


0.20 


4 


0.45 


0.15 


5 


0.35 


0.10 


6 


0.25 


0.05 



III. Full Graph Message-Passing Algorithm 

The decoder uses the sum-product algorithm to compute the maximum a posteriori probability estimate of the 
codeword given the channel output. The graph on which the algorithm operates represents the following factorization: 

p(X|R) oc p(R|X)P(X) 

« l[p(r(i,j)\Mk,l) : (k,l)GAf(i,j)})P(X), (3) 

(i,3) 

where X and R are matrices representing the channel input and channel output, respectively. Af(i,j) is as defined 
previously. -P(X) is the probability that X is a codeword of the LDPC code being used. This probability can be 
represented graphically via the Tanner graph of the LDPC code [22]. p(R|X) represents the ISI channel. 

Fig. shows an illustration of this factor graph where nearest neighbor interference is assumed. This "full graph" 
has three types of nodes: variable nodes, check nodes, and measured data nodes corresponding to the codeword 
bits, the parity-check equations, and the observed data symbols, respectively. The upper two levels in the full graph 
represent the LDPC code bipartite graph showing how the codeword bits are connected to the check nodes via the 
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LDPC code parity-check matrix. The lower two levels represent the channel ISI graph showing how the ISI induces 
dependencies between the codeword bits. For clarity, connections for only one measured data node are shown in 
the figure. 




Fig. 2. The full graph depicting the factorization of ^3). 

Message-passing on this full graph is performed using the following schedule of messages: variable nodes to 
check nodes, check nodes to variable nodes, variable nodes to measured data nodes and finally measured data nodes 
to variable nodes. Following is a brief description of how the messages are updated for each of the aforementioned 
four steps for the sum-product algorithm. The messages in the update equations are probabilities. 
Variable-to-check messages: The message from a variable node x to a check node c at the Zth iteration is calculated 
using the messages passed to x from its neighboring check and measured data nodes at the (I — l)th iteration. 

Miic(Q) = a n ^(o) n ^(o) 

rEN r (x) c'£N c (x)\c 

/4ic(i) = a n ^ja) n (4) 

r£N r (x) c'eN c (x)\c 

where ^i'l +c (-), ^r^Lx(-), and ^l£Lx{-) are, respectively, the variable-to-check, measured data-to-variable, and check- 
to-variable messages at the Zth iteration. N c (x) and N r (x) are the neighboring check and measured data nodes of 
x respectively, and a is a normalizing constant. 

Check-to-variable messages: At the Zth iteration the check-to-variable messages are calculated using the variable- 
to-check messages at the Zth iteration and the sum-product rule, 
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4ix(0) - ^P(c|x = 0,5c c ) J] A&Lc&O 

x c x'6iV(e)\x 

^(1) = pffi^LXe) J] ^tc(^) (5) 
x c x'6iV(e)\x 

where iV(c) are the variable nodes connected to check node c; x c are length \N(c) | — 1 binary tuples. The conditional 
probabilities above are or 1 depending on whether the parity-check constraint at node c is satisfied or not. 
Variable-to-measured data messages: The variable-to-measured data messages at the Zth iteration are calculated 
using the messages received at the variable nodes from the check nodes at the Zth iteration and from the measured 
data nodes at the (I — l)th iteration, 

/4i,(a) = « II <"c-Uo) II ^1>) 

c£N c (x) r'£N r (x)\r 

Mxlr(l) = « II ^lx(l) II (6) 

c£N c (x) r'£N r (x)\r 

Measured data-to-variable messages: To complete one iteration, the measured data-to-variable messages at the 
Zth iteration are computed using the variable-to-measured data messages at the Zth iteration and the sum-product 
rule, 

,41,(0) = aj^p(r\x = 0,5c r )^ r (x') 

,41,(1) = aJ2p(r\x = l,5i r )^t r (x') (7) 

N(r) is the set of all variable nodes connected to measured data node r; x r are length 7V(r) — 1 binary tuples. The 
conditional probabilities above are values of a Gaussian probability density function. The mean of this probability 
density function is determined by the signal levels given in Table |I] and the variance is equal to the noise variance 
a 2 . 

After this step the "pseudo-posterior" probabilities are calculated using the messages from the check nodes and 
the measured data nodes, 

?i°(o) = « n n ^cix(o) 

r£N r (x) c£N c (x) 

4 ;) (i) = « n ^i,(i) n Miix(i). (8) 

r£N r (x) ceJVc(x) 

The codeword estimate is obtained by setting x to 1 if <7x^(l) > Qx\o) and otherwise. The decoding stops if 
the decoder converges to a codeword or a maximum number of iterations are exhausted. The complexity of the full 
graph algorithm is linear in the LDPC code block length and quadratic in the size of the interference neighborhood. 
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Fig. 3. Results of using the full graph message-passing algorithm for the TWODOS recording model given by and Table □ 
The LDPC code used is a block length 10000 regular (3,30) code. 

Results of using full graph message-passing for the TWODOS channel model of (|2ji and Table U are shown in 
Fig- E The LDPC code used is a block length 10000, regular (3,30) code [23]. This high-rate code is chosen so 
as to add only a small number of redundant parity bits. The SNR is defined as the average energy in the signal 
divided by the noise power; 

V 6 ( e )(s 2 +s 2 ) 
SNR = ioi oglo ^|^^I Zd , (9) 

where s no (s ni ) is the signal level given that the central bit is a 0(1) and has n nonzero neighbors and R is the 
LDPC code rate. From right to left the solid curves in Fig. [3] correspond to the performance of the full graph for 
a maximum of 1, 2, 3, 4, and 5 iterations. The number of iterations is kept low so as to reduce decoding delay. 
The dashed curves correspond to the uncoded performance which is the performance when the full graph algorithm 
iterates only between equations (jfji and ignoring the LDPC code. This then gives a low-complexity detection 
scheme for the nonlinear ISI channel. As the curves show, the improvement in performance is quite significant. At 
a bit error rate of 1G -5 the coded performance after 5 iterations is about 8 dB better than the uncoded performance 
after 10 iterations. Also, the performance of the full graph algorithm after 5 iterations is about 8 dB better (at a 
bit error rate of 10 -6 ) than the performance reported in [15], where a stripe-wise Viterbi detector is used for the 
same ISI. 
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IV. Density Evolution and Threshold Computation 

For many channels and decoders of interest, LDPC codes exhibit a threshold phenomenon [23]; there exists a 
critical value of the channel parameter (noise tolerance threshold, say 6*) such that an arbitrarily small bit error 
probability can be achieved if the noise level (<5) is smaller than 5* and the code length is long enough. On the other 
hand, for S > 5* the probability of bit error is larger than a positive constant. Richardson and Urbanke [23] developed 
an algorithm called density evolution for iteratively calculating message densities, enabling the determination of 
the aforementioned threshold. Kavcic et al., [21] extended the work of Richardson and Urbanke to compute noise 
tolerance thresholds for one-dimensional ISI channels. 

Using a density evolution similar to that proposed by Kavcic et al, [21] noise tolerance thresholds are computed 
for the full graph algorithm and the nonlinear TWODOS channel. Following is a brief description of the algorithm. 
Density evolution tracks the "evolution" of the probability density function (pdf) of correct (or incorrect) messages 
passed on the graph as the iterations progress. Density evolution is described more conveniently using log-likelihood 
ratios (LLR) for messages instead of probabilities. The LLR for the variable-to-check messages at the Zth iteration 
is defined as La?l c = log^f 4 ^-. The LLRs L c ^ x , L x ^ r , and L r ^ x are defined analogously. Using LLRs gives 

M«— *c(U 

an equivalent representation of the full graph message-passing algorithm. Equations (|4j-Q can be rewritten as 



41 c = E 41i } + E 4^2 ( 10 > 

r£N r (x) c'eN c (x)\c 

tanh(^p) = (-l) c [] tanh (^) dD 

x'eN(c)\x 

41, = E 41*+ E 412 d2) 

cGN c (x) r'eN r (x)\r 

41, = F(L«t r ;x'eN(r)\x), (13) 

The update for measured data-to-variable messages has no closed form when represented using LLRs. The "tanh" 
rule [22] cannot be applied to {7) since the measured data nodes are not binary-valued. Equation dl 3I > represents 
the measured data-to-variable message update via a function F which performs the appropriate computation. 

Let f}; (x) be the pdf of the correct message from a variable node to a check node at the Ith round of message- 
passing. This pdf is evolved through (II 0I-(I1 31. The evolution of the density functions through fllOi and Jl 2i are 
simple convolutions and can be implemented efficiently using the fast Fourier transform. Density evolution for (II It 
can be implemented using the change in measure described by Richardson and Urbanke in [23] or more efficiently 
by using a table-lookup as explained by Chung et al., in [24]. For density evolution through (1131 Monte Carlo 
simulations are used; message-passing is performed on the channel graph using a long block length and the pdf 
of the outgoing messages from the measured data nodes is approximated by using the histogram of computed 
messages. 
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After evolving fP(x) through (Hol-iDl. fi l+1) (x) is obtained. Using fv (x), the error probability at the 
(I + l)th iteration, pi l+1 \ can be computed as 

Pi l+1) = f fi l+1) (x)dx. (14) 

The noise tolerance threshold, 5*, can be calculated as the supremum of all S for which the error probability 
goes to zero as the iterations progress. Table UTI shows the computed thresholds for regular LDPC codes of different 
rates. The (3, oo) code refers to the case when no coding is used. The noise tolerance threshold is the variance of 
the AWGN. The SNR is calculated as in except that the rate of the code is not taken into account. 

TABLE II 

Noise tolerance thresholds for the TWODOS channel. 



LDPC 
Code 


Code 
Rate 


Threshold 

ol 


Threshold 
SNR [dB] 


(3,3) 


0.000 


0.0670 


1.7270 


(3,4) 


0.250 


0.0436 


3.5929 


(3,5) 


0.400 


0.0283 


5.4699 


(3,6) 


0.500 


0.0215 


6.6633 


(3,9) 


0.667 


0.0140 


8.5264 


(3,12) 


0.750 


0.0117 


9.3059 


(3,15) 


0.800 


0.0103 


9.8593 


(3,30) 


0.900 


0.0061 


12.1344 


(3,60) 


0.950 


0.0035 


14.5470 


(3,90) 


0.967 


0.0030 


15.2165 


(3,120) 


0.975 


0.0027 


15.6741 


(3,150) 


0.980 


0.0025 


16.0083 


(3,oo) 


1.000 


0.0018 


17.4342 



In proving the existence of thresholds for memoryless channels the crucial innovation of Richardson and Urbanke 
in [23] were the "concentration results." These results state that as the block length tends to infinity the performance 
of the LDPC decoder on random graphs converges to its expected behavior and that the expected behavior can 
be determined from the corresponding cycle-free behavior. Kavcic et al, [21] extended these concentration results 
to one-dimensional ISI channels by using LDPC coset codes to circumvent the complications arising out of the 
input-dependent memory. 

For 2D ISI channels the concentration results do not hold since the channel graph has short cycles even in the 
limit of infinitely long block length. Hence existence of thresholds cannot be proved using the concentration analysis. 
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Fig. 4. Results of using the full graph message-passing algorithm for the TWODOS recording model using different length 
regular (3,30) LDPC codes. 

However, our simulations suggest that for the TWODOS channel the full graph algorithm respects the thresholds 
computed using density evolution. These results are shown in Fig. 0] for three block lengths: 10000, 65536, and 
262144 and using a regular (3,30) LDPC code. The results show that very low bit error rates are obtained only 
when the noise variance is smaller than the threshold. Although this does not prove the existence of a threshold, 
it suggests that the noise tolerance thresholds of Table |ll| are upper bounds on the performance of the full graph 
algorithm. Besides that, the thresholds also serve as a design parameter; given a system with a specified SNR it 
is sufficient to pick an LDPC code having a smaller threshold SNR thereby ensuring that the bit-error rate can be 
made arbitrarily small as the block length increases. 

V. Conclusions 

A message-passing based scheme for joint equalization and decoding for nonlinear two-dimensional intersymbol 
interference channels has been proposed. The scheme, called the full graph algorithm, performs sum-product 
message-passing on a joint graph of the error correction code and the channel. The complexity of the full graph 
algorithm is linear in the block length of the error correction code and quadratic in the size of interference 
neighborhood. The performance of the algorithm is studied for the two-dimensional optical storage paradigm. 
Simulations for the nonlinear channel model of TWODOS show significant improvement over uncoded performance. 
The performance is about 6 to 8 dB better than that reported in by Immink et al, [15] for the same intersymbol 
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interference. A detection scheme for nonlinear ISI channels has also been proposed. This scheme performs sum- 
product message-passing on the graph corresponding to the nonlinear channel ISI. Using density evolution noise 
tolerance thresholds for the full graph algorithm are also computed and are shown to accurately predict the 
performance of the algorithm as the block length gets large. 
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